Search for: All records

ABSTRACT: The Consensual Assessment Technique (CAT) is one of the most effective and commonly used design evaluation methods. However, it fails to capture implicit cognitive processes and has mainly been studied in a homogenous design modality. To bridge this gap, the present study investigates the impact of design ideas represented in different modalities (i.e., text-only, sketch-only, text + sketch) on design evaluations for creativity, novelty, and usefulness, and examine human gaze patterns during the evaluation process. Our findings showed that novice raters exhibit higher interrater reliability and greater convergence in visual attention when rating ideas containing sketches compared to text-only design modality, highlighting the value of visual elements in design evaluations. 

Abstract Engineering design is undergoing a transformative shift with the advent of AI, marking a new era in how we approach product, system, and service planning. Large language models have demonstrated impressive capabilities in enabling this shift. Yet, with text as their only input modality, they cannot leverage the large body of visual artifacts that engineers have used for centuries and are accustomed to. This gap is addressed with the release of multimodal vision-language models (VLMs), such as GPT-4V, enabling AI to impact many more types of tasks. Our work presents a comprehensive evaluation of VLMs across a spectrum of engineering design tasks, categorized into four main areas: Conceptual Design, System-Level and Detailed Design, Manufacturing and Inspection, and Engineering Education Tasks. Specifically in this paper, we assess the capabilities of two VLMs, GPT-4V and LLaVA 1.6 34B, in design tasks such as sketch similarity analysis, CAD generation, topology optimization, manufacturability assessment, and engineering textbook problems. Through this structured evaluation, we not only explore VLMs’ proficiency in handling complex design challenges but also identify their limitations in complex engineering design applications. Our research establishes a foundation for future assessments of vision language models. It also contributes a set of benchmark testing datasets, with more than 1000 queries, for ongoing advancements and applications in this field. 

Text-to-image models are enabling efficient design space exploration, rapidly generating images from text prompts. However, many generative AI tools are imperfect for product design applications as they are not built for the goals and requirements of product design. The unclear link between text input and image output further complicates their application. This work empirically investigates design space exploration strategies that can successfully yield product images that are feasible, novel and aesthetic – three common goals in product design. Specifically, users’ actions within the global and local editing modes, including their time spent, prompt length, mono versus multi-criteria prompts, and goal orientation of prompts, are analyzed. Key findings reveal the pivotal role of mono versus multi-criteria and goal orientation of prompts in achieving specific design goals over time and prompt length. The study recommends prioritizing the use of multi-criteria prompts for feasibility and novelty during global editing while favoring mono-criteria prompts for aesthetics during local editing. Overall, this article underscores the nuanced relationship between the AI-driven text-to-image models and their effectiveness in product design, urging designers to carefully structure prompts during different editing modes to better meet the unique demands of product design. 

Text-to-image generative models have increasingly been used to assist designers during concept generation in various creative domains, such as graphic design, user interface design, and fashion design. However, their applications in engineering design remain limited due to the models’ challenges in generating images of feasible designs concepts. To address this issue, this paper introduces a method that improves the design feasibility by prompting the generation with feasible CAD images. In this work, the usefulness of this method is investigated through a case study with a bike design task using an off-the-shelf text-to-image model, Stable Diffusion 2.1. A diverse set of bike designs are produced in seven different generation settings with varying CAD image prompting weights, and these designs are evaluated on their perceived feasibility and novelty. Results demonstrate that the CAD image prompting successfully helps text-to-image models like Stable Diffusion 2.1 create visibly more feasible design images. While a general tradeoff is observed between feasibility and novelty, when the prompting weight is kept low around 0.35, the design feasibility is significantly improved while its novelty remains on par with those generated by text prompts alone. The insights from this case study offer some guidelines for selecting the appropriate CAD image prompting weight for different stages of the engineering design process. When utilized effectively, our CAD image prompting method opens doors to a wider range of applications of text-to-image models in engineering design. 

Abstract Sketch2Prototype is an AI-based framework that transforms a hand-drawn sketch into a diverse set of 2D images and 3D prototypes through sketch-to-text, text-to-image, and image-to-3D stages. This framework, shown across various sketches, rapidly generates text, image, and 3D modalities for enhanced early-stage design exploration. We show that using text as an intermediate modality outperforms direct sketch-to-3D baselines for generating diverse and manufacturable 3D models. We find limitations in current image-to-3D techniques, while noting the value of the text modality for user-feedback. 

Abstract Understanding relationships between different products in a market system and predicting how changes in design impact their market position can be instrumental for companies to create better products. We propose a graph neural network-based method for modeling relationships between products, where nodes in a network represent products and edges represent their relationships. Our modeling enables a systematic way to predict the relationship links between unseen products for future years. When applied to a Chinese car market case study, our method based on an inductive graph neural network approach, GraphSAGE, yields double the link prediction performance compared to an existing network modeling method—exponential random graph model-based method for predicting the car co-consideration relationships. Our work also overcomes scalability and multiple data type-related limitations of the traditional network modeling methods by modeling a larger number of attributes, mixed categorical and numerical attributes, and unseen products. While a vanilla GraphSAGE requires a partial network to make predictions, we augment it with an “adjacency prediction model” to circumvent the limitation of needing neighborhood information. Finally, we demonstrate how insights obtained from a permutation-based interpretability analysis can help a manufacturer understand how design attributes impact the predictions of product relationships. Overall, this work provides a systematic data-driven method to predict the relationships between products in a complex network such as the car market. 

Abstract Graph representation learning has revolutionized many artificial intelligence and machine learning tasks in recent years, ranging from combinatorial optimization, drug discovery, recommendation systems, image classification, social network analysis to natural language understanding. This paper shows their efficacy in modeling relationships between products and making predictions for unseen product networks. By representing products as nodes and their relationships as edges of a graph, we show how an inductive graph neural network approach, named GraphSAGE, can efficiently learn continuous representations for nodes and edges. These representations also capture product feature information such as price, brand, and engineering attributes. They are combined with a classification model for predicting the existence of a relationship between any two products. Using a case study of the Chinese car market, we find that our method yields double the F-1 score compared to an Exponential Random Graph Model-based method for predicting the co-consideration relationship between cars. While a vanilla Graph-SAGE requires a partial network to make predictions, we augment it with an ‘adjacency prediction model’ to circumvent this limitation. This enables us to predict product relationships when no neighborhood information is known. Finally, we demonstrate how a permutation-based interpretability analysis can provide insights on how design attributes impact the predictions of relationships between products. Overall, this work provides a systematic method to predict the relationships between products in a complex engineering system. 

Statistical network models have been used to study the competition among different products and how product attributes influence customer decisions. However, in existing research using network-based approaches, product competition has been viewed as binary (i.e., whether a relationship exists or not), while in reality, the competition strength may vary among products. In this paper, we model the strength of the product competition by employing a statistical network model, with an emphasis on how product attributes affect which products are considered together and which products are ultimately purchased by customers. We first demonstrate how customers’ considerations and choices can be aggregated as weighted networks. Then, we propose a weighted network modeling approach by extending the valued exponential random graph model to investigate the effects of product features and network structures on product competition relations. The approach that consists of model construction, interpretation, and validation is presented in a step-by-step procedure. Our findings suggest that the weighted network model outperforms commonly used binary network baselines in predicting product competition as well as market share. Also, traditionally when using binary network models to study product competitions and depending on the cutoff values chosen to binarize a network, the resulting estimated customer preferences can be inconsistent. Such inconsistency in interpreting customer preferences is a downside of binary network models but can be well addressed by the proposed weighted network model. Lastly, this paper is the first attempt to study customers’ purchase preferences (i.e., aggregated choice decisions) and car competition (i.e., customers’ co-consideration decisions) together using weighted directed networks.